There are not guarantees or assurances provided, all examples are selected for demonstration purposes.
Copyright (c) 2020-2021 by Gustavo Zambrana
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
import os
print( os.environ['PYTHONPATH'] )
%reload_ext autoreload
%autoreload 2
# Python ≥3.6 is required
import sys
assert sys.version_info >= (3, 6)
# Common imports
import numpy as np
import os
import pandas as pd
import yfinance as yf
import feather
import tables
import ta
from datetime import datetime
# Ignore useless warnings (see SciPy issue #5998)
import warnings
warnings.filterwarnings(action="ignore", message="^internal gelsd")
import sqlalchemy
print(sqlalchemy.__version__)
from sqlalchemy.orm import with_polymorphic
from trade_engine.persist.DBLayer import DBLayer
from trade_engine.persist.DBSchema import DBSchema
import logging
from logging.config import fileConfig
fileConfig('/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/log/logging.ini')
#logger = logging.getLogger('dev')
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger().setLevel(logging.INFO)
myDBLayer = DBLayer()
# ***
# *** Connect String for SQL Lite db in memory
# ***
# sqliteConnString = 'sqlite:///:memory:'
# myDBLayer.connect( connString = sqliteConnString )
# ***
# *** Connect String for SQL Lite db file
# ***
# dbTradeEngTestPath = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/db/dbTradeEngTest.db'
# ConnString = 'sqlite:///' + dbTradeEngTestPath
# ***
# *** Postgres DB
# ***
ConnString = 'postgresql+pg8000://postgres:postgres@localhost:7777/trade_engine_dev_db'
# *** Connect to DB, Create DB Schema if not already created, and acquired a DB session
myDBLayer.connect( connString = ConnString )
DBSchema.createDBSchema()
session = DBLayer.getSession()
DBSchema.dropAll()
The purpose of the engine is:
Performs a backtest for a single trade strategy or a portfolio of trade strategies
Support different kinds of backtests and optimizations of hyper-parameters per trade strategy
Analyzed out-of-sample performance for trade strategies and/or portfolio
Support Technical Analysis Models, as well as AI Based Models to manage all trade actions, that is:
Enter, Exit, Position Sizing, Stop-Loss, Trail-Stop-Loss, etc.
Technical Analysis Models are modelts that:
AI Based Models are any models that:
Support ensemble models for any trade action in a Trade Strategy, i.e.:
Support mixing Technical Analysis and AI Based Models in ensembles
Support persisting all information to execute a backtest in a Relational Database, as well as the results
Setup the initial architecture to enable distributed cloud execution of backtests (future)
There are different types of backtest supported, the following is a list of the different backtests types and their objectives:
**1. Backtest**
**2. WFA Backtest**
**3. Sliding Window WFA Backtest**
**4. Grid-Search Backtest**
**5. Production-Sim Backtest**
This section gives an overview of the current examples provided, and general instructions
The examples are categorized as follows:
There is a common set of imports for all examples at the beginning that is required to be imported
The examples should be reviewed sequentially, as they get more complex and present more features
as they progress. In addition, some examples refered to explanations and models from previous ones.
Every example is self-contained, this leads to a bit of duplication of code. However,in the interest
of maximizing clarity this approached is used.
Every example has a detail description of the backtest at the start, the trade strategies in the
backtest, the trade actions configured and their supporting models, their Fit Periods (only required
if at least one AI model is present in the backtest), Test Period, etc.
Saving and Re-Storing any objects to a database is optional, in some examples is presented, but it is
the same procedure for all examples/objects. In all cases, saving an object, cascades and saves
all children objects, and retreiving an object, retrieves all children objects (lazy loading).
Refer to SQLAlchemy for additional details.
The Trade Engine objects keep state associated with a given backtest, the only objects that should
be shared between backtests are Market Data Objects, and Pipeline Data Managers, but they don't have
to be.
Examples:
**Backtest**
1. Nasdaq ETF - Yahoo Finance Market Data - SMA Trade Strategy
2. Cottons Futures - Quandal Market Data - SMA Trade Strategy
3. Nasdaq ETF - Yahoo Finance Market Data - LSTM Trade Strategy
4. Portfolio A: Long and SMA Trade Strategies for US Indixes
5. Portfolio B: Long and AI LSTM based Strategies for US Indixes
6. Portfolio C: Long and Multiple Configured SMA Trade Strategies for US Indixes
7. Portfolio D: Long and Multiple Configured LSTM Trade Strategies for US Indixes
8. Portfolio E: Long and Eleven Distinct Technical Analysis Based Trade Strategies for US Indixes
9. Portfolio F: Long and Multiple Configured AI Modeled Trade Strategies for US Indixes
10. Portfolio G: All Trade Strategies in Portfolio E and F
11. Portfolio H: Utilized Leveraged ETFs for Trading on The Model Trade-Asset
12. Portfolio I: Monthly Rebalance of Funds in a Portfolio
13. Portfolio J: Trade Strategies with different Market Data Sources (Quandal, YahooFinance, User-Defined DataFrame)
**WFA Backtest**
1. WFA-Backtest-1
2. WFA-Backtest-2
**Slide Window WFA Backtest**
1. SW-WFA-Backtest 1
2. SW-WFA-Backtest 2
3. SW-WFA-Backtest 3
**Grid Backtest**
To be added
**Production Sim Backtest**
To be added
**Other**
To be added
# *** Imports required for running Trade Strategy examples
from trade_engine.model.TradeStrategyBaseModelPersist import TradeStrategyBaseModelPersist
from trade_engine.model.TradeStrategyMarketDataMng import TradeStrategyMarketDataMng
from trade_engine.model.MarketDataPeriod import MarketDataPeriod
from trade_engine.ts_actions.generate.TradeStrategyActionGenerator import TradeStrategyActionGenerator
from trade_engine.ts_actions.generate.TradeStrategyConfigData import TradeStrategyConfigData
from trade_engine.investment_manager.TradingAssetInvMngBase import TradingAssetInvMngBase
from trade_engine.registry.TradingAssetExecutionType import TradingAssetExecutionType
from trade_engine.registry.TradingAssetRegisterItem import TradingAssetRegisterItem
from trade_engine.broker.SimBrokerTrading import SimBrokerTrading
from trade_engine.execution.BacktestExecution import BacktestExecution
from trade_engine.dataset_def.DatasetDef import DatasetDef, DatasetType
from trade_engine.dataset_def.FitDatasetsDef import FitDatasetsDef
from trade_engine.reports.BacktestReports import BacktestReports
# *** Additional Imports required for running Portfolio examples
from tradeg.market_data.MarketData import MarketData
from trade_engine.model.TALibPipelineBuilder import TALibPipelineBuilder
from trade_engine.model.TradeStrategyModelPipelineDataMng import TradeStrategyModelPipelineDataMng
from trade_engine.enter_ts.TradeEnterClassicStrategy import TradeEnterClassicStrategy
from trade_engine.exit_ts.TradeExitClassicStrategy import TradeExitClassicStrategy
from trade_engine.model.classic.CacheTradingModel import CacheTradingModel
from trade_engine.model.classic.TradingRulesModel import TradingRulesModel
from trade_engine.model.classic.VotingTradingRulesModel import VotingTradingRulesModel
from trade_engine.exit_ts.TradeExitStack import TradeExitStack
from trade_engine.enter_ts.TradeEnterStack import TradeEnterStack
from trade_engine.investment_manager.PortfolioInvMng import PortfolioInvMng
from trade_engine.investment_manager.PortfolioRebalanceFrequency import PortfolioRebalanceFrequency
from trade_engine.registry.PortfolioRegisterItem import PortfolioRegisterItem
from trade_engine.model.ai.LSTM_Model import LSTM_Model
from trade_engine.model.ai.WaveNet_RNN_AI_TradingModel import WaveNet_RNN_AI_TradingModel
from trade_engine.model.ai.WaveNet_RNN_AI_TradingModel import WaveNet_RNN_AI_Trading_Rules
from trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder import WaveNet_RNN_AI_PipelineBuilder
from trade_engine.enter_ts.TradeEnterAIStrategy import TradeEnterAIStrategy
from trade_engine.exit_ts.TradeExitAIStrategy import TradeExitAIStrategy
from trade_engine.ts_actions.policy.TradeEnterSetPolicy import TradeEnterSetPolicy, TradeEnterSetVotingPolicy
from trade_engine.ts_actions.policy.TradeEnterPolicy import TradeEnterPolicy
from trade_engine.ts_actions.policy.TradeExitSetPolicy import TradeExitSetPolicy, TradeExitSetVotingPolicy
from trade_engine.ts_actions.policy.TradeExitPolicy import TradeExitPolicy
# *** Additional Imports required for running WFA Backtest and Slide Win WFA Backtest
from trade_engine.execution.BacktestExecutionWFA import BacktestExecutionWFA, WFASlideWindowPeriod
from trade_engine.execution.BacktestExecutionSlideWindowWFA import BacktestExecutionSlideWindowWFA
from trade_engine.execution.BacktestExecutionSlideWindowWFA import WFASlideWinScenarioFixDatasets
from trade_engine.execution.BacktestExecutionSlideWindowWFA import WFASlideWinScenarioSlidingDatasets
from trade_engine.execution.BacktestExecutionSlideWindowWFA import WFASlideWinScenarioAnchoredSlidingDatasets
SMA Trade Strategy for Nasdaq ETF
Description:
1. $1.0M Budget
2. 100% capital invested
Trade Strategies:
1. One trade strategy: Nasdaq using Simple Moving Average(SMA) for entering/exiting a trade
Backtest:
1. Type: regular
2. Test Period:
1. Begin Date = 2001-01-01
2. End Date = 2005-12-31
trade_assets = [ "^IXIC" ]
Market Data Manager configured to use yahoo-finance for obtaining market data
The following TradeStrategyMarketDataMng object will do the following:
1. Download market data for:
[ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
2. Used yahoo-finance as the market provider
3. Save the downloaded data in the directory and file specified by the arguments:
1. dirPathMarketData
2. fileNameMarketData
4. It will request the market data provider to get data from "beg_date_str" to "end_date_str"
5. If end_date_str is not defined (as the case below), all data available from the begin date is
requested.
6. Interval is set to "1d", this parameter is market data provider specific, see Yahoo-Finance,
Quandal for what options they support
7. The **Adj Close** column is configured as the default price column
8. Note that even so leveraged ETFs are requested from 2000-01-01 to the present, the data
actually returned for these stock symbols by Yahoo-Finance doesn't start until 2011.
The behavior is market data provider specific
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_marketdat_mng.marketDataObj.getDataFrame()
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# SMA JSON Configuration
#
# Model Configuration:
# win_fast : fast moving average
# win_slow : slow moving average
# Signals to enter and exit a trade based on the fast ma crossing the slow ma
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 20, 'win_slow' : 100 } },
} } )
tsActionGen = TradeStrategyActionGenerator()
trade_asset = trade_assets[0]
# *** Trade Strategy Entry and Exit Rules based on SMA
ts_unique_name = "ts_sma"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat )
# *** Trade Strategy Model
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0, #invest all funds avail in each trade
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
trade_enter_set.tsEntries[0].ts_model_pipline_mng.\
getAllModelDataPipelineDataForStockSymbol(stockSymbol = "^IXIC" )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Nasdaq ETF - SMA Trade Strategy",
timePeriod = DatasetDef(datasetType = DatasetType.Testing,
strBegDate = "2001-01-01",
strEndDate = "2005-12-31" ),
trade_asset_registered_item = ta_registered_item )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
SMA Trade Strategy for Cottons Future
Description:
1. $1.0M Budget
2. 100% capital invested
Trade Strategies:
1. One trade strategy: Cotton's Futures using Simple Moving Average(SMA) for entering/exiting a trade
Backtest:
1. Type: regular
2. Test Period:
1. Begin Date = 2002-01-01
2. End Date = 2005-12-31
Download the following data from Quandal:
CHRIS/ICE_CT1
Where CHRIS is the Quandal Data Provider, and the name after the / is the Futures symbol.
The format of the JSON to download from Quandal is:
{ quandal-path :
[ description of data,
[ columns to extract from the returned Quandal DataFrame ],
{ "Quandal_Column" : "Rename Quandal Column" }
]
}
Note:
1. The nested dictionary to rename the original downloaded Quandal data, it is provided since Quandal does
not have standard output column names.
- Futures data from different data sources might have different returned column names for the same
state, that is "Close" vs. "close", etc.
- For sharing the same market data manager object it is recommended to rename the columns to have
standard names, and match the ones provided by Yahoo Finance
2. Yahoo Finance returns standard equity column names
3. All implemented AI and Technical Models used standard column names (same as Yahoo Finance)
Columns: Adj Close, Close, Open, High, Low, Volume
Index: Datetime
4. In order to use the implemented models the Quandal Market Data or any Custom Market Data must have
similar named columns, or the user can implement his own models and use whatever column names he/she
prefers.
5. In this example, the Quandal data source does not provide all standard column names, only open,
which is re-named to 'Adj Close'. Therefore only some of the built models will be available,
as they required High, Low, such as Bollinger Bands
indexQuantCodeMap = {
"CHRIS/ICE_CT1" : ["Cotton", ["Open", "Volume"], { "Open" : "Adj Close" } ]
}
dirPathMarketDataQuandl = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/TSMarketDat'
beg_date_for_feature_creation = "2000-10-01"
beg_date_str = "2001-01-01"
end_date_str = "2010-12-31"
q_stock_index_list = list ( indexQuantCodeMap.keys() )
ts_marketdat_mng_quandal = TradeStrategyMarketDataMng(\
marketDataProviderName = "quandl",
supportedTradeAssets = q_stock_index_list,
stock_price_col_name = "Adj Close",
fileNameMarketData = "Quandl_Test_1",
dirPathMarketData = dirPathMarketDataQuandl,
beg_date_str = beg_date_for_feature_creation,
end_date_str = end_date_str,
interval = "1d" ,
updateDataPassEndDate = False,
forceCreateMarketData = False,
indexQuantCodeMap = indexQuantCodeMap # required for quandl market data provider, defined above
)
ts_marketdat_mng_quandal.marketDataObj.getDataFrame()
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# SMA JSON Configuration
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 20, 'win_slow' : 100 } },
} } )
tsActionGen = TradeStrategyActionGenerator()
trade_asset = "CHRIS/ICE_CT1"
# *** Trade Strategy Entry and Exit Rules based on SMA
ts_unique_name = "ts_sma"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng_quandal,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat )
# *** Trade Strategy Model
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng_quandal,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0, #invest all funds avail in each trade
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
trade_enter_set.tsEntries[0].ts_model_pipline_mng.\
getAllModelDataPipelineDataForStockSymbol(stockSymbol = trade_asset )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Cottons Futures - SMA Trade Strategy",
timePeriod = DatasetDef(datasetType = DatasetType.Testing,
strBegDate = "2002-01-01",
strEndDate = "2005-12-31" ),
trade_asset_registered_item = ta_registered_item )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
LSTM Trade Strategy for Nasdaq ETF
Description:
1. $1.0M Budget
2. 100% capital invested
Trade Strategies:
1. One trade strategy: Nasdaq using a LSTM-Model for entering/exiting a trade
Backtest:
1. Type: regular
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-06-30
4. Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2008-12-31
trade_assets = [ "^IXIC" ]
Market Data Manager configured to use yahoo-finance for obtaining market data
The following TradeStrategyMarketDataMng object will do the following:
1. Download market data for:
[ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
2. Used yahoo-finance as the market provider
3. Save the downloaded data in the directory and file specified by the arguments:
1. dirPathMarketData
2. fileNameMarketData
4. It will request the market data provider to get data from "beg_date_str" to "end_date_str"
5. If end_date_str is not defined (as the case below), all data available from the begin date is
requested.
6. Interval is set to "1d", this parameter is market data provider specific, see Yahoo-Finance,
Quandal for what options they support
7. The **Adj Close** column is configured as the default price column
8. Note that even so leveraged ETFs are requested from 2000-01-01 to the present, the data
actually returned for these stock symbols by Yahoo-Finance doesn't start until 2011.
The behavior is market data provider specific
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_marketdat_mng.marketDataObj.getDataFrame()
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# LSTM JSON Configuration
#
# Model configuration:
# Simple LSTM MOdel with two layers
# n_steps : length of time series
# no_neurons_lay1 : layer 1 neurons
# no_neurons_lay2 : layer 2 neurons
# pred_ahead_periods : Number of prediction points ahead
# epochs : Number of epochs
# ext_trade_dyn : Extend an existent trade dymically based on current period predictions
# re_eval_exit_dyn : Re-evaluate exiting a trade dynamically based on current period predictions
# entry_policy_args :
# lastn_match : Enter a trade if the last n predictions matched
# cross_enter_signal : Enter a trade if the there is a cross-over signal,
# from buy-to-sell, sell-to-buy
# exit_policy_args :
# lastn_match : Exit a trade if the last n predictions matched
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
tsActionGen = TradeStrategyActionGenerator()
trade_asset = "^IXIC"
# *** Trade Strategy Entry and Exit Rules based on SMA
ts_unique_name = "ts_lstm"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat )
# *** Trade Strategy Model
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0, #invest all funds avail in each trade
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
# Pipeline just normalizes the data
trade_enter_set.tsEntries[0].ts_model_pipline_mng.\
getAllModelDataPipelineDataForStockSymbol(stockSymbol = "^IXIC" )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2000-07-01",
strBegDate = "2001-01-01",
strEndDate = "2005-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-06-30" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2008-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Nasdaq-ETF LSTM based Trade Strategy",
timePeriod = testing_period,
fitDatasets = fit_datasets,
trade_asset_registered_item = ta_registered_item )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio A: Long and SMA Trade Strategies for US Indixes
Description:
1. Portfolio with 6 trading strategies
2. $1.2M Budget
3. No-Rebalance during the backtest
4. 100% capital invested
5. Portfolio investment using equal weights for each trade strategy
Trade Strategies:
1. Three trade strategies: Nasdaq, S&P 500, and DOW Jones with buy-and-hold
2. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using SMA
Backtest:
1. Type: regular
2. Test Period:
1. Begin Date = 2001-01-01
2. End Date = 2001-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
This is a second more detail approach to built a SMA trade strategy versus the example SMA Test Trade Strategy
json_dat = {
"version" : "1.0",
"indicators" : {
"SMA" : {
"ta_indicators" : {
"sma_20": {
"package" : "ta.trend",
"class_name" : "SMAIndicator",
"input_param_series" : ["close"],
"input_param_series_mkd_col": ["Adj Close"],
"input_params_other" : { "window": 20, "fillna": False },
"output_pipeline_cols" : [ "sma_20" ],
"output_generating_funcs" : [ "sma_indicator()" ]
},
"sma_100": {
"package" : "ta.trend",
"class_name" : "SMAIndicator",
"input_param_series" : ["close"],
"input_param_series_mkd_col": ["Adj Close"],
"input_params_other" : { "window": 100, "fillna": False },
"output_pipeline_cols" : [ "sma_100" ],
"output_generating_funcs" : [ "sma_indicator()" ]
}
},
"signals" : {
"sma20_gt_sma100": {
"columns" : [ "sma_20", "sma_100" ],
"relationship" : "gt",
"signal_col" : "sma20_gt_sma100"
},
"sma20_lt_sma100": {
"signal_gen_func" : "signalSampleCreator",
"func_args" : { },
"signal_col" : "sma20_lt_sma100"
},
}
},
},
}
def signalSampleCreator(df : pd.DataFrame, **kwargs ) -> pd.core.series.Series :
return df["sma_20"] < df["sma_100"]
taPipilineObj = TALibPipelineBuilder( trade_assets = portfolio_trade_assets,
json_dat = json_dat,
signalgen_func_dict = { 'signalSampleCreator' : signalSampleCreator } )
ts_model_pipline_mng = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
modelPipelineBuilder = taPipilineObj,
fileNameMarketData = "US_Indixes_And_ETFs_PipelineData",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
dfx = ts_model_pipline_mng.getAllModelDataPipelineDataForStockSymbol(stockSymbol = "^GSPC" )
dfx
brokerTradingObj = SimBrokerTrading()
# Conducting a backtest for a Trade Strategy requires:
#
# 1. A Trade Strategy Model
# - A TS Model is associated with only on trade asset
# - References a Market Data Manager
# - Trading Rules: Entry, Exit,Stop-Loss, Trail-Stop-Loss, etc. (only Entry rules are required)
# - Each Trading Action may have a Pipeline Data Manager associated with.
# - For example: For using SMA to enter/exit trades, the signasl must be computed based on the
# Market Data and SMA indicators/signals configured
# - Different trade actions might share the same pipeline manager or create different onces
# - Each Trading Action has their own configuration/metadata
# - Entry Rules:
# Decide how to combine signals and which signals to enter/exit
# Entry Policies:
# - Only enter a trade if the last n dates/periods were marked for entering a trade
# - Only enter a trade if there is a cross-over signal (from buy to sell, or sell to buy)
# More complex configuration options are availabe for creating entry rules based on an ensemble
# of entry rule algorithms. This will be explained and detail in other examples.
#
# 2. A Broker to conduct the trades with
# - can be shared across Trade Strategies, but no required
#
# 3. A Trade Strategy Investment Manager
# - Manages the budget and spending rules (max/min to invest)
# - Trades
# - Interfaces with Broker objects to place, cancel, alter trades
# - Other investment metadata rules
# - Works with Portfolio Investment Managers to rebalance funds across a portfolio
#
# 4. A Trade Asset Register Item
# - This is a compose object to have all necessary information to conduct a backtest
# either as a single item, or as part of a portfolio
# - The dates and characteristics of the backtest execution are defined separately
# Each Trade Register Item Object has:
# - A Name
# - One Trade Strategy
# + Each Trade Strategy has one or more actions defined (only entry rules are mandatory)
# a) Entry Rules
# b) Exit Rules
# c) Stop Loss and/or Trail Stop Rules
# d) Position Sizing Rules per trade
# etc.
# - One Investment Manager
# - A broker to trade with (simulation, or paper trading)
taRegisterItems = [ ]
for trade_asset in portfolio_trade_assets :
# *** Trading Rules Model, generating buy, sell, hold trade decisions based on long/short signals
# *** N number of signals can be configured to entered long or short trades, the
# *** long_signals_bool_logic or short_signals_bool_logic control determine logic to apply
# *** See also: VotingTradingRulesModel() to derive rules based on voting metadata
classic_model_enter = TradingRulesModel( name = "SMA_Enter",
enter_long_signals = [ "sma20_gt_sma100" ],
enter_short_signals = [ "sma20_lt_sma100" ],
long_signals_bool_logic = "and",
short_signals_bool_logic = "and" )
classic_model_exit = TradingRulesModel( name = "SMA_Exit",
enter_long_signals = [ "sma20_gt_sma100" ],
enter_short_signals = [ "sma20_lt_sma100" ],
long_signals_bool_logic = "and",
short_signals_bool_logic = "and" )
# *** TS Entry Rules
# *** Uses Trading Rules Model configured above to determine when to enter a trade
ts_entry = TradeEnterClassicStrategy(
classic_trading_model = CacheTradingModel( name = "SMA_20_AND_100_Enter",
classic_model = classic_model_enter ),
ts_model_pipline_mng = ts_model_pipline_mng )
# *** TS Exit Rules
# *** Uses Trading Rules Model configured above to determine when to exit a trade,
# *** The exit rules can used a different set of signals (EMA, BB, or AI based rules) than entry rules,
# *** in this example the entry and exit rules are based on the same set of SMA trading rules
ts_exit = TradeExitClassicStrategy(
classic_trading_model = CacheTradingModel( name = "SMA_20_AND_100_Exit",
classic_model = classic_model_exit ),
ts_model_pipline_mng = ts_model_pipline_mng )
# *** Trade Strategy
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = TradeEnterStack( tsEntries = [ ts_entry ] ),
ts_exit = TradeExitStack( tsExits = [ ts_exit ] ),
model_type = "sma",
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "TA - SMA for: {0}".format(trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - SMA Algo",
initInvestment = 12000000.0,
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio A, US-Indexes using SMA and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio A, US-Indexes using SMA and Long strategies" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
tsExecuteEngine = BacktestExecution( \
name = "Backtest for SMA based Trade Strategies",
timePeriod = DatasetDef(datasetType = DatasetType.Testing,
strBegDate = "2001-01-01",
strEndDate = "2001-12-31" ),
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del tsExecuteEngine
for tsObj in session.query(BacktestExecution):
print(tsObj.name)
if tsObj.name == "Backtest for SMA based Trade Strategies" :
break
tsExecuteEngine = tsObj
print(tsExecuteEngine.id, tsExecuteEngine.name)
tsExecuteEngine.getExecutionMetrics()
Portfolio B: Long and AI LSTM based Strategies for US Indixes
Description:
1. Portfolio with 6 trading strategies
2. $1.2M Budget
3. No-Rebalance during the backtest
4. 100% capital invested
5. Portfolio investment using equal weights for each trade strategy
Trade Strategies:
1. Three trade strategies: Nasdaq, S&P 500, and DOW Jones with buy-and-hold
2. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using LSTM AI Model
Backtest:
1. Type: regular
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-06-30
4. Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2008-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_model_pipline_mng_b = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_B",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
ts_model_pipline_mng_b.getAllModelDataPipelineDataForStockSymbol(stockSymbol = "^GSPC" )
brokerTradingObj = SimBrokerTrading()
# Conducting a backtest for a Trade Strategy requires:
#
# 1. A Trade Strategy Model
# - A TS Model is associated with only on trade asset
# - References a Market Data Manager
# - Trading Rules: Entry, Exit,Stop-Loss, Trail-Stop-Loss, etc. (only Entry rules are required)
# - Each Trading Action may have a Pipeline Data Manager associated with.
# - For example: For using SMA to enter/exit trades, the signasl must be computed based on the
# Market Data and SMA indicators/signals configured
# - Different trade actions might share the same pipeline manager or create different onces
# - Each Trading Action has their own configuration/metadata
# - Entry Rules:
# Decide how to combine signals and which signals to enter/exit
# Entry Policies:
# - Only enter a trade if the last n dates/periods were marked for entering a trade
# - Only enter a trade if there is a cross-over signal (from buy to sell, or sell to buy)
# More complex configuration options are availabe for creating entry rules based on an ensemble
# of entry rule algorithms. This will be explained and detail in other examples.
#
# 2. A Broker to conduct the trades with
# - can be shared across Trade Strategies, but no required
#
# 3. A Trade Strategy Investment Manager
# - Manages the budget and spending rules (max/min to invest)
# - Trades
# - Interfaces with Broker objects to place, cancel, alter trades
# - Other investment metadata rules
# - Works with Portfolio Investment Managers to rebalance funds across a portfolio
#
# 4. A Trade Asset Register Item
# - This is a compose object to have all necessary information to conduct a backtest
# either as a single item, or as part of a portfolio
# - The dates and characteristics of the backtest execution are defined separately
# Each Trade Register Item Object has:
# - A Name
# - One Trade Strategy
# + Each Trade Strategy has one or more actions defined (only entry rules are mandatory)
# a) Entry Rules
# b) Exit Rules
# c) Stop Loss and/or Trail Stop Rules
# d) Position Sizing Rules per trade
# etc.
# - One Investment Manager
# - A broker to trade with (simulation, or paper trading)
taRegisterItems = [ ]
for trade_asset in portfolio_trade_assets :
# LSTM Trading Model, Configure with Trading Rules
#
# There are two parts two configured an AI Trading Model:
#
# 1) Defined the Actual AI Architecture and configured it:
# - for this example it is an LSTM model with two layers, the model params are customed
# - This LSTM network uses two x-features for predictions, and predicts 5 steps ahead
# - AI Models are derived from the class : AI_ModelBase
#
#. 2) Defined the Trading Rules for an AI Model
# - How predictions from an AI model translated to invest/no-invest signals, and
# trade type (long or short)
# - AI Models are derived from the class : AI_TradingModelBase
# - The WaveNet_RNN_AI_TradingModel class has a generic set of trading rules for DL
# time series forecasting models
# - The Trading Model feeds the Pipeline data built for the AI model in tensor format
#
# - Configured the trading model metadata
# - x-features : 'Adj Close', 'Volume' { for predicting , could be more or less}
# - y-label : 'Adj Close' { just predict adj close }
# - creates time series that are 50 steps long for predicting
# - The AI time series model will predict 5 steps ahead
#
# - The following trading metadata rules are defined with the following params:
# a) trad_pred_rule =
# i) WaveNet_RNN_AI_Trading_Rules.n_pred_ahead
# { use n days ahead prediction } or
# ii) WaveNet_RNN_AI_Trading_Rules.n_pred_mean
# { predict using the mean of n days ahead }
# b) trad_pred_dat = [ n ] { predict using n periods ahead, 0 = Next Period }
# c) extend_trade_exit_dyn = [ True, False ]
# Yes, an open trade will be extended based on current forecast of data
# ex: If a trade was opened in a long position with 5 days ahead prediction,
# if option is enable, every period, the trade will be analyzed for extending
# d) re_evaluate_forecast_exit_dyn = [ True, False ]
# True = exit trade earlier than first forecast, base on new data
# ex: if a trade was opened in a long position using 5 days ahead forecast,
# if this option is enable, the trade can be exited after any period,
# if the LSTM Model starts predicting a different signal/trade-type than the
# one entered.
#
num_x_features = 2
num_y_predict_ahead_periods = 5
ai_model = LSTM_Model( epochs = 20,
num_x_features = num_x_features,
num_y_predictions = num_y_predict_ahead_periods,
lstm_layers_neuros = [ 20, 20 ], # number of LSTM stack layers, and thier num of neurons
optimizer = "adam",
activation = "relu",
rand_seed = 42 )
ai_trading_model = WaveNet_RNN_AI_TradingModel(\
name = "LSTM_{0}".format(trade_asset),
ai_model = ai_model,
num_x_features = num_x_features,
features = ['Adj Close', 'Volume'],
features_label = 'Adj Close',
n_steps = 50,
n_pred_ahead = num_y_predict_ahead_periods,
trad_pred_rule = WaveNet_RNN_AI_Trading_Rules.n_pred_ahead,
trad_pred_dat = [ 0 ],
extend_trade_exit_dyn = True,
re_evaluate_forecast_exit_dyn = True )
# *** TS Entry Rules
# *** Uses AI Trading Model configured above to determine when to enter a trade
ts_enter_ai = TradeEnterAIStrategy(name = "Enter_LSTM",
ai_trading_model = ai_trading_model,
ts_model_pipline_mng = ts_model_pipline_mng_b )
ts_enter_ai_set = TradeEnterStack( tsEntries = [ ts_enter_ai ],
tsPolicy = TradeEnterSetPolicy( \
lastn_match = 1,
votingPolicy = TradeEnterSetVotingPolicy.Majority )
)
# *** TS Exit Rules
# *** Uses AI Trading Model configured above to determine when to enter a trade
ts_exit_ai = TradeExitAIStrategy(name = "Exit_LSTM",
ai_trading_model = ai_trading_model,
ts_model_pipline_mng = ts_model_pipline_mng_b )
ts_exit_ai_set = TradeExitStack( tsExits = [ ts_exit_ai] )
# *** Trade Strategy
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = ts_enter_ai_set,
ts_exit = ts_exit_ai_set,
model_type = "lstm",
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "TA - LSTM for: {0}".format(trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - LSTM Algo",
initInvestment = 12000000.0,
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio B, US-Indexes using LSTM and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio B, US-Indexes using LSTM and Long strategies" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2000-07-01",
strBegDate = "2001-01-01",
strEndDate = "2005-12-30" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-06-30" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2008-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for LSTM based Trade Strategies",
timePeriod = testing_period,
fitDatasets = fit_datasets,
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del tsExecuteEngine
for tsObj in session.query(BacktestExecution):
print(tsObj.name)
if tsObj.name == "Backtest for LSTM based Trade Strategies" :
break
tsExecuteEngine = tsObj
print(tsExecuteEngine.id, tsExecuteEngine.name)
tsExecuteEngine.getExecutionMetrics()
Portfolio C: Long and Multiple Configured SMA Trade Strategies for US Indixes
Description:
1. Portfolio with 27 trading strategies
2. $200,000/strategy, Total Budget = $5.4M
3. No-Rebalance during the backtest
4. 100% capital invested
5. Portfolio investment using equal weights for each trade strategy
Trade Strategies:
1. Three trade strategies: Nasdaq, S&P 500, and DOW Jones with buy-and-hold
2. 21 trade strategies: Nasdaq, S&P 500, and DOW Jones using following SMA Configuration:
sma-conf fast_mav, slow_mav, enter_if_last_nmatch, enter_if_cross_over_sig, exit_if_last_nmatch
1 20 100 2 True 2
2 10 50 2 True 2
3 10 50 2 True 1
4 15 60 2 True 2
5 15 75 2 True 1
6 7 35 2 True 2
7 7 35 2 True 1
3. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using the following configuration:
- A Composite SMA Trade Strategy based on the the first five SMA Configured trade strategies
above
- The Set Entry cofiguration rules are:
a. Majority Voting (default)
Three of the Five SMA Indicators configured in the set must signaled the same type of
investment decision.
b. Enter a trade if the last n voting decisions are the same, set to 1 (default )
c. Enter a trade if there is cross-over signaled after applying rules a) and b),
set to False (default)
- The Set Exit configuration rules are:
a. Majority Voting (Default)
- Exit a long trade if the percent of SMA Indicators signaling an exit for a long position
is greater that the percent signaling no-exit
- Exit a short trade if the percent of SMA Indicators signaling an exit for a short position
is greater that the percent signaling no-exit
b. Exit a trade if the last n exit decisions are the same, set to 1 (default )
4. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using the following configuration:
- A Composite SMA Trade Strategy based on all the SMA Configured trade strategies
- The Set Entry cofiguration rules are:
a. Voting Percent
The majority of the signals indicating an investment position (either short or long),
must also meet a minimum percent of 50% across all the strategies in the set.
b. Enter a trade if the last n voting decisions are the same, set to 1 (default )
c. Enter a trade if there is cross-over signaled after applying rules a) and b),
set to False (default)
- The Set Exit configuration rules are:
a. Majority Voting (Default)
- Exit a long trade if the percent of SMA Indicators signaling an exit for a long position
is greater that the percent signaling no-exit
- Exit a short trade if the percent of SMA Indicators signaling an exit for a short position
is greater that the percent signaling no-exit
b. Exit a trade if the last n exit decisions are the same, set to 1 (default )
Backtest:
1. Type: regular
2. Test Period:
1. Begin Date = 2001-01-01
2. End Date = 2001-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 20, 'win_slow' : 100 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_3",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_4",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_5",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 75 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #6
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_6",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 7, 'win_slow' : 35 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #7
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_7",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 7, 'win_slow' : 35 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = None )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = None )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Multiple-SMA TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio C, US-Indexes with Multiple-SMA and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio C, US-Indexes with Multiple-SMA and Long strategies" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio of Multiple-SMA based Trade Strategies",
timePeriod = DatasetDef(datasetType = DatasetType.Testing,
strBegDate = "2001-01-01",
strEndDate = "2001-12-31" ),
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']]
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio D: Long and Multiple Configured LSTM Trade Strategies for US Indixes
Description:
1. Portfolio with 27 trading strategies
2. $200,000/strategy, Total Budget = $5.4M
3. No-Rebalance during the backtest
4. 100% capital invested
5. Portfolio investment using equal weights for each trade strategy
Trade Strategies:
1. Three trade strategies: Nasdaq, S&P 500, and DOW Jones with buy-and-hold
2. 21 trade strategies: Nasdaq, S&P 500, and DOW Jones using following LSTM Configurations:
LSTM Models
Parameter #1 #2 #3 #4 #5 #6 #7
epochs 20 20 20 20 20 20 20
no neurons layer 1 20 20 20 20 20 20 20
no neurons layer 2 20 20 20 20 20 20 20
time series length 20 100 50 50 40 30 60
no predictions ahead 3 15 5 5 3 3 5
extend trade dyn Yes Yes Yes Yes Yes Yes Yes
re_evalaute_exit_dyn Yes Yes Yes Yes Yes Yes Yes
enter_if_last_nmatch 1 1 1 1 1 1 1
enter_if_cross_over_sig No No No No No No No
exit_if_last_nmatch 1 1 1 2 1 1 1
3. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using the following Composite
model configuration:
- A Composite LSTM Trade Strategy based on all the LSTM configured trade strategies above
- The Set Entry cofiguration rules are:
a. Majority Voting (default)
b. Enter a trade if the last n voting decisions are the same, set to 1 (default )
c. Enter a trade if there is cross-over signaled after applying rules a) and b),
set to False (default)
- The Set Exit configuration rules are:
a. Majority Voting (Default)
- Exit a long trade if the percent of LSTM Indicators signaling an exit for a long position
is greater that the percent signaling no-exit
- Exit a short trade if the percent of LSTM Indicators signaling an exit for a short position
is greater that the percent signaling no-exit
b. Exit a trade if the last n exit decisions are the same, set to 1 (default )
Backtest:
1. Type: regular
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-06-30
4. Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2008-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 100, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 15,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_3",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_4",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_5",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #6
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_6",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 30, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #7
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_7",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 60, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = None )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = None )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Multiple-LSTM TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio D, US-Indexes with Multiple-LSTM and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio D, US-Indexes with Multiple-LSTM and Long strategies" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2000-07-01",
strBegDate = "2001-01-01",
strEndDate = "2005-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-06-30" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2008-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio of Multiple-LSTM based Trade Strategies",
timePeriod = testing_period,
fitDatasets = fit_datasets,
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']]
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio E: Long and Eleven Distinct Technical Analysis Based Trade Strategies for US Indixes
Description:
1. Portfolio with 39 trading strategies
2. $200,000/strategy, Total Budget = $7.8M
3. No-Rebalance during the backtest
4. 100% capital invested
5. Portfolio investment using equal weights for each trade strategy
Trade Strategies:
1. Three trade strategies: Nasdaq, S&P 500, and DOW Jones with buy-and-hold
2. 33 trade strategies: Nasdaq, S&P 500, and DOW Jones using following TA Indicators & Configurations:
See: https://technical-analysis-library-in-python.readthedocs.io/en/latest/index.html for
explanation on the technical indicator specific parameters
TS-No TA-Indicator Parameters
1 sma fast_mav 10
slow_mav 50
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
2 bb window 20
window_dev 1
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
3 macd window_slow 26
window_fast 12
window_sign 9
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
4 rsi enter_long if rsi <= 30
exit_long if rsi > 30
enter_short if rsi >= 70
exit_short if rsi <= 60
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
5 srsi window 14
smooth1 3
smooth2 3
enter_long if stochrsi_d >= 0.5
exit_long if stochrsi_d < 0.5
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
6 adx window 14
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
7 mkama kama_fast_window 10
kama_fast_pow1 2
kama_fast_pow1 30
kama_slow_window 10
kama_slow_pow1 5
kama_slow_pow1 30
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
8 tsi window_slow 25
window_fast 5
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
9 forceidx window 13
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
10 mfi window 14
no mfi points for slope 5
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
11 sma fast_mav 15
slow_mav 60
enter_if_last_nmatch 2
enter_if_cross_over_sig Yes
exit_if_last_nmatch 1
3. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using the following configuration:
- A Composite Trade Strategy based on all the technical-analysis indicators configured
above
- The Set Entry cofiguration rules are:
a. Majority Voting (default)
b. Enter a trade if the last n voting decisions are the same, set to 1 (default )
c. Enter a trade if there is cross-over signaled after applying rules a) and b),
set to False
- The Set Exit configuration rules are:
a. Majority Voting (Default)
b. Exit a trade if the last n exit decisions are the same, set to 1 (default )
Backtest:
1. Type: regular
2. Test Period:
1. Begin Date = 2006-01-01
2. End Date = 2010-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_bb",
'actions_conf' : { "entry_exit" :
{ 'model' : "bb",
'type' : "ta",
'obj_args' : { 'window' : 13, 'window_dev' : 1 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_macd",
'actions_conf' : { "entry_exit" :
{ 'model' : "macd",
'type' : "ta",
'obj_args' : { 'window_slow' : 26, 'window_fast' : 12, 'window_sign' : 9,
'enter_long_macd_diff' : 0.5 , 'exi_long_macd_diff' : 0.5,
'enter_short_macd_diff' : -0.5, 'exi_short_macd_diff' : -0.5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_rsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "rsi",
'type' : "ta",
'obj_args' : { 'rsi_enter_long' : 30.0, 'rsi_exit_long' : 30.0,
'rsi_enter_short' : 70.0, 'rsi_exit_short' : 70.0 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #6
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_srsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "srsi",
'type' : "ta",
'obj_args' : { 'window' : 14, 'smooth1' : 3, 'smooth2' : 3,
'srsi_enter_long' : 0.8, 'srsi_exit_long' : 0.8,
'srsi_enter_short' : 0.2, 'srsi_exit_short' : 0.2 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #7
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_adx",
'actions_conf' : { "entry_exit" :
{ 'model' : "adx",
'type' : "ta",
'obj_args' : { 'window' : 14, 'adx_enter_long_ge' : 14, 'adx_exit_long_lt' : 14,
'adx_enter_short_ge' : 14, 'adx_exit_short_lt' : 14 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_kama",
'actions_conf' : { "entry_exit" :
{ 'model' : "kama",
'type' : "ta",
'obj_args' : { 'window_fast' : 10, 'pow1_fast' : 2, 'pow2_fast' : 30,
'window_slow' : 10, 'pow1_slow' : 5, 'pow2_slow' : 30 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #9
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_tsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "tsi",
'type' : "ta",
'obj_args' : { 'window_slow' : 25, 'window_fast' : 5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #10
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_forceindex",
'actions_conf' : { "entry_exit" :
{ 'model' : "forceindex",
'type' : "ta",
'obj_args' : { 'window' : 39 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #11
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_mfi",
'actions_conf' : { "entry_exit" :
{ 'model' : "mfi",
'type' : "ta",
'obj_args' : { 'window' : 14, 'long_slope_win' : 15, 'short_slope_win' : 15 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = None )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = None )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured TA Register Items : ", len(taRegisterItems ))
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Distinct TechAnal TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with Distinct TechAnal & Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio of Distinct TechAnal & Long Strategies",
timePeriod = DatasetDef(datasetType = DatasetType.Testing,
strBegDate = "2006-01-01",
strEndDate = "2010-12-31" ),
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']]
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio F: Long and Multiple Configured AI-Modeled Trade Strategies for US Indixes
Description:
1. Portfolio with 30 trading strategies
2. $200,000/strategy, Total Budget = $6M
3. No-Rebalance during the backtest
4. 100% capital invested
5. Portfolio investment using equal weights for each trade strategy
Trade Strategies:
1. Three trade strategies: Nasdaq, S&P 500, and DOW Jones with buy-and-hold
2. 24 trade strategies: Nasdaq, S&P 500, and DOW Jones using the following AI-Models
and corresponding configurations:
Trading AI-Models
Parameter #1 #2 #3 #4 #5 #6 #7
ai-model GRU LSTM Conv1d-GRU Conv1d-LSTM Wavenet Wavenet-GRU GRU
epochs 20 20 20 20 20 20 20
no neurons layer 1 20 20 20 20 - 20 20
no neurons layer 2 20 20 20 20 - 20 20
time series length 50 50 50 50 50 50 40
no predictions ahead 5 5 5 5 5 5 3
extend trade dyn Yes Yes Yes Yes Yes Yes Yes
re_evalaute_exit_dyn Yes Yes Yes Yes Yes Yes Yes
enter_if_last_nmatch 1 1 1 1 1 1 1
enter_if_cross_over_sig No No No No No No No
exit_if_last_nmatch 1 1 1 1 1 1 1
Parameter #8
ai-model Wavenet-LSTM
epochs 20
no neurons layer 1 20
no neurons layer 2 20
time series length 50
no predictions ahead 5
extend trade dyn Yes
re_evalaute_exit_dyn Yes
enter_if_last_nmatch 1
enter_if_cross_over_sig No
exit_if_last_nmatch 1
Note:
- the parameters "no neurons layer 1" and "no neurons layer 2" are only applicable to
AI Models that have GRU or LSTM layers.
- Conv1d is a convulation layer in one dimension (configured with 40 filters )
- Wavenet is an AI Architectured based on a set of Convolutional Layers
3. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using the following Composite
model configuration:
- A Composite Trade Strategy based on all the AI Trading Models configured above
- The Set Entry cofiguration rules are:
a. Majority Voting (default)
b. Enter a trade if the last n voting decisions are the same, set to 1 (default )
c. Enter a trade if there is cross-over signaled after applying rules a) and b),
set to False (default)
- The Set Exit configuration rules are:
a. Majority Voting (Default)
- Exit a long trade if the percent of Trading AI Models signaled an exit for a long position
is greater that the percent signaling no-exit
- Exit a short trade if the percent of Trading AI Models signaled an exit for a short position
is greater that the percent signaling no-exit
b. Exit a trade if the last n exit decisions are the same, set to 1 (default )
Backtest:
1. Type: regular
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2002-01-01
2. End Date = 2006-12-31
2. Validation Period:
1. Begin Date = 2007-01-01
2. End Date = 2008-06-30
3. Test Period:
1. Begin Date = 2009-01-01
2. End Date = 2009-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_model_pipline_mng_f = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_D",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
# Define Metadata for several Trading AI-Models for several Trade Strategies in the portfolio and one
# Trade Strategy based on a composite set of all Trading AI-Models. The composite trade strategy
# uses Majority Voting for entering and exiting a trade (default setting)
# The configuraiton data for each Trade Strategy is defined via a JSON/dict, and
# they are added to the TradeStrategyConfigData() object
#
# This configuration method is a short-cut when creating many trade strategies in a portfolio, or
# a composite of many trade strategies. The JSON format follows the general format:
#
# *** Trade Strategy Config:
#
# 1. ts_unique_tag : unique tag to identify a trade strategy in a set
# 2. actions_conf : dict,
# Description: contains the configuration data for each action in a trade strategy, only
# the "entry_exit", sub-tag is required
# Child tags:
# "entry_exit" : required
# "stop_lost" : optional
# "pos_sizing" : optional
# "pyramiding" : optional
# 3. The format of each child-tag/action follows the same parameters as when defined via the class,
# 4. obj_args : custom set of parameters passed to the defined model defined via the model tag, this
# is a dynamic set of parameters and specific to each model, some share common inputs.
#
# Supported built-in models, for the entry_exit action:
# [ "sma", "bb", "macd", "rsi", "srsi", "adx", "kama", "tsi",
# "forceindex", "mfi", 'conv1d_gru', 'conv1d_lstm', 'wavenet',
# 'wavenet_gru', 'wavenet_lstm', 'lstm', 'gru' ]
#
# Custom models can be added via a custom function. The second strategy in the configuration
# below ('ts_unique_tag' : "ts_gru_1") , defines the "custom_create_func" tag and sets equal to
# "create_actions_ai_ts_model", this function should be already defined or imported. Thus for
# trade strategy #2, the user custom function is used. Note, that for trade strategy: ts_gru_2
# the model is also set to "gru", but in this case it is using the built-in gru model.
# The actual implementantions for all supported models are included in this Jupyter Notebook, at
# the beg. of this portfolio example, and a user can copy and altered as they see fit, it would
# affect the built-in functions, which are scope in a pythong package.
#
#
# *** Trade Strategy Set Config:
#
# There is only one set configuration, and it is defined to apply policies on how to treat the
# set actions, this configuration is optional, and defaults for each action are defined below
#
# "entry" : optional
# Defines the policy to drive the trade decision to follow from the set. As explained in earlier
# examples.
# Default policy parameters are:
# Majority Voting,
# incNoInvestInVotingDec = False,
# lastn_match = 1
# cross_over_signal_for_trading
#
# "exit" : optional
# Defines the policy to drive the exit decision to follow from the set. As explained in earlier
# examples.
# Default policy parameters are:
# Majority Voting,
# lastn_match = 1
#
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_wavenet",
'actions_conf' : { "entry_exit" :
{ 'model' : "wavenet",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #6
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_wavenet_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "wavenet_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #7
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_wavenet_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "wavenet_lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_f )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_f )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Multiple-AI-Trading Models TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio F, US-Indexes with Mult-AI-Trading and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio F, US-Indexes with Mult-AI-Trading and Long strategies" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2001-07-01",
strBegDate = "2002-01-01",
strEndDate = "2006-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2006-07-01",
strBegDate = "2007-01-01",
strEndDate = "2008-06-30" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2008-07-01",
strBegDate = "2009-01-01",
strEndDate = "2009-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio of Multiple-AI-Modeled based Trade Strategies",
timePeriod = testing_period,
fitDatasets = fit_datasets,
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']]
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio G: Combines all Trade Strategies in Portfolio E and F
Description:
1. Portfolio with 63 trading strategies
2. $200,000/strategy, Total Budget = $12.6M
3. No-Rebalance during the backtest
4. 100% capital invested
5. Portfolio investment using equal weights for each trade strategy
Trade Strategies:
1. Three trade strategies: Nasdaq, S&P 500, and DOW Jones with buy-and-hold
2. Fifty-Seven trade strategies, 19 trade strategies per US Index, as defined in
Portfolio E and F
3. Three trade strategies: Nasdaq, S&P 500, and DOW Jones using the following Composite
model configuration:
- A Composite Trade Strategy based on all the Trading Models configured above per US Index
- The Set Entry cofiguration rules are:
a. Majority Voting (default)
b. Enter a trade if the last n voting decisions are the same, set to 1 (default )
c. Enter a trade if there is cross-over signaled after applying rules a) and b),
set to False (default)
- The Set Exit configuration rules are:
a. Majority Voting (Default)
- Exit a long trade if the percent of Trading Models signaled an exit for a long position
is greater that the percent signaling no-exit
- Exit a short trade if the percent of Trading Models signaled an exit for a short position
is greater that the percent signaling no-exit
b. Exit a trade if the last n exit decisions are the same, set to 1 (default )
Backtest:
1. Type: regular
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2006-12-31
2. Validation Period:
1. Begin Date = 2007-01-01
2. End Date = 2008-12-31
3. Test Period:
1. Begin Date = 2009-01-01
2. End Date = 2009-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_model_pipline_mng_g = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_G",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_wavenet",
'actions_conf' : { "entry_exit" :
{ 'model' : "wavenet",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #6
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_wavenet_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "wavenet_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #7
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_wavenet_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "wavenet_lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #9
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #10
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #11
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_bb",
'actions_conf' : { "entry_exit" :
{ 'model' : "bb",
'type' : "ta",
'obj_args' : { 'window' : 13, 'window_dev' : 1 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #12
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_macd",
'actions_conf' : { "entry_exit" :
{ 'model' : "macd",
'type' : "ta",
'obj_args' : { 'window_slow' : 26, 'window_fast' : 12, 'window_sign' : 9,
'enter_long_macd_diff' : 0.5 , 'exi_long_macd_diff' : 0.5,
'enter_short_macd_diff' : -0.5, 'exi_short_macd_diff' : -0.5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #13
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_rsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "rsi",
'type' : "ta",
'obj_args' : { 'rsi_enter_long' : 30.0, 'rsi_exit_long' : 30.0,
'rsi_enter_short' : 70.0, 'rsi_exit_short' : 70.0 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #14
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_srsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "srsi",
'type' : "ta",
'obj_args' : { 'window' : 14, 'smooth1' : 3, 'smooth2' : 3,
'srsi_enter_long' : 0.8, 'srsi_exit_long' : 0.8,
'srsi_enter_short' : 0.2, 'srsi_exit_short' : 0.2 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #15
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_adx",
'actions_conf' : { "entry_exit" :
{ 'model' : "adx",
'type' : "ta",
'obj_args' : { 'window' : 14, 'adx_enter_long_ge' : 14, 'adx_exit_long_lt' : 14,
'adx_enter_short_ge' : 14, 'adx_exit_short_lt' : 14 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #16
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_kama",
'actions_conf' : { "entry_exit" :
{ 'model' : "kama",
'type' : "ta",
'obj_args' : { 'window_fast' : 10, 'pow1_fast' : 2, 'pow2_fast' : 30,
'window_slow' : 10, 'pow1_slow' : 5, 'pow2_slow' : 30 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #17
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_tsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "tsi",
'type' : "ta",
'obj_args' : { 'window_slow' : 25, 'window_fast' : 5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #118
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_forceindex",
'actions_conf' : { "entry_exit" :
{ 'model' : "forceindex",
'type' : "ta",
'obj_args' : { 'window' : 39 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #19
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_mfi",
'actions_conf' : { "entry_exit" :
{ 'model' : "mfi",
'type' : "ta",
'obj_args' : { 'window' : 14, 'long_slope_win' : 15, 'short_slope_win' : 15 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_g )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_g )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - TechAnal & AI Trading Models TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio G, US-Indexes with TechAnal & AI Trading Models, and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio G, US-Indexes with TechAnal & AI Trading Models, and Long strategies" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2000-07-01",
strBegDate = "2001-01-01",
strEndDate = "2006-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2006-07-01",
strBegDate = "2007-01-01",
strEndDate = "2008-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2008-07-01",
strBegDate = "2009-01-01",
strEndDate = "2009-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio G",
timePeriod = testing_period,
fitDatasets = fit_datasets,
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
DOW Jones
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("DJI")', engine='python')
Nasdaq
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("IXIC")', engine='python')
S&P 500
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("GSPC")', engine='python')
Dow Jones
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^DJI" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^DJI" ').T.head(60)
Nasdaq
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^IXIC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^IXIC" ').T.head(60)
S&P 500
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^GSPC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^GSPC" ').T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio H: Utilized Leveraged ETFs for Trading on The Model Security
- All models are trained on their corresponding US Index/model, but the trade enter via a broker is with
the underlying leveraged ETF.
- ETF data for US Indixes not available until 2011 in yahoo finance
- Five Trade Strategies are executed with ETF on, and Five with ETF off to show effect of ETF trading
- Additionally, there is a composite trade strategy for all ETF strategies and one for non-ETF
trade strategies
Backtest:
1. Type: regular
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2007-01-01
2. End Date = 2012-12-31
2. Validation Period:
1. Begin Date = 2013-01-01
2. End Date = 2014-12-31
3. Test Period:
1. Begin Date = 2015-01-01
2. End Date = 2015-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
MarketData.getDateRangesForStocks( ts_marketdat_mng.marketDataObj.getDataFrame() )
ts_model_pipline_mng_h = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_H",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_bb",
'actions_conf' : { "entry_exit" :
{ 'model' : "bb",
'type' : "ta",
'obj_args' : { 'window' : 13, 'window_dev' : 1 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_macd",
'actions_conf' : { "entry_exit" :
{ 'model' : "macd",
'type' : "ta",
'obj_args' : { 'window_slow' : 26, 'window_fast' : 12, 'window_sign' : 9,
'enter_long_macd_diff' : 0.5 , 'exi_long_macd_diff' : 0.5,
'enter_short_macd_diff' : -0.5, 'exi_short_macd_diff' : -0.5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_rsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "rsi",
'type' : "ta",
'obj_args' : { 'rsi_enter_long' : 30.0, 'rsi_exit_long' : 30.0,
'rsi_enter_short' : 70.0, 'rsi_exit_short' : 70.0 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
# **** Configure ETF based Trade Strategies
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_h )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_h )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
trade_on_ETFs = True,
trade_assets_etf_map = { "^DJI" : [ "UDOW", "SDOW"],
"^IXIC" : [ "TQQQ", "SQQQ"],
"^GSPC" : [ "SPXL", "SPXS"] },
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}_etf".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
# **** Configure Non-ETF based Trade Strategies (for comparison)
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_h )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_h )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
trade_on_ETFs = False,
trade_assets_etf_map = { "^DJI" : [ "UDOW", "SDOW"],
"^IXIC" : [ "TQQQ", "SQQQ"],
"^GSPC" : [ "SPXL", "SPXS"] },
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Using ETFs for Trading",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio H, US-Indexes - Using ETFs for Trading, and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio H, US-Indexes - Using ETFs for Trading, and Long strategies" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2006-07-01",
strBegDate = "2007-01-01",
strEndDate = "2012-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2012-07-01",
strBegDate = "2013-01-01",
strEndDate = "2014-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2014-07-01",
strBegDate = "2015-01-01",
strEndDate = "2015-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio H",
timePeriod = testing_period,
fitDatasets = fit_datasets,
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
DOW Jones
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("DJI")', engine='python')
Nasdaq
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("IXIC")', engine='python')
S&P 500
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("GSPC")', engine='python')
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^DJI"' )
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^IXIC"' )
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^GSPC"' )
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio I: Monthly Rebalance of Funds in the Portfolio
- Same strategies as defined and explained in Portfolio C.
- Only difference is the Portfolio Investment Manager is configured to balance funds monthly,
using an equal weights policy.
Backtest:
1. Type: regular
2. Test Period:
1. Begin Date = 2012-01-01
2. End Date = 2015-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 20, 'win_slow' : 100 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_3",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_4",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_5",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 75 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #6
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_6",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 7, 'win_slow' : 35 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #7
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_7",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 7, 'win_slow' : 35 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = None )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = None )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Montly-Rebalance TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Monthly, # Monthly Rebalance
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio Inv. Manager for US-Indexes - Montly-Rebalance TS",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio Inv. Manager for US-Indexes - Montly-Rebalance TS" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio I",
timePeriod = DatasetDef(datasetType = DatasetType.Testing,
strBegDate = "2012-01-01",
strEndDate = "2015-12-31" ),
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']]
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Portfolio J: Trade Strategies with different Market Data Sources (Quandal, YahooFinance, User-Defined DataFrame)
- Futures data is sourced from Quandal (Cotton, Soy Beans, and S&P 500 futures)
- US Equity indexes from Yahoo-Finance
- Leverage ETFs TQQQ and SQQQ from the user-defined DataFrame
- Multiple SMA Models, and LSTM, GRU, MACD, and RSI Models defined to use with each trading asset
- Additionally, there is a composite trade strategy for all Futures, and one composite trade
strategy for the US Index equities
- Buy-and-Hold strategies for each trade asset for comparison
Backtest:
1. Type: regular
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2006-12-31
2. Validation Period:
1. Begin Date = 2007-01-01
2. End Date = 2008-06-30
3. Test Period:
1. Begin Date = 2009-01-01
2. End Date = 2009-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "2000-01-01"
end_date_str = None
ts_marketdat_mng_yf = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_marketdat_mng_yf.marketDataObj.getDataFrame()
User can provide their own market data via a Pandas DataFrame
Requirements:
1. DataFrame Index : must be called "Date", and be of datetime format
2. Index column : All data associated with a given trade asset. Same exact format as the
yahoofinance above
3. The rest of the columns in the dataframe can be named anything. However, it is recommended
they used standard names as the ones in YahooFinance above; this will make all built-in
AI and Technical Analysis Models available.
4. Note, interval, supportedTradeAssets are constraint and not enforced; the data that is passed
in the dataframe is what is supported.
ts_marketdat_mng_df = TradeStrategyMarketDataMng(\
marketDataProviderName = "dataframe",
marketDataDF = ts_marketdat_mng_yf.marketDataObj.getDataFrame(),
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_DF",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_marketdat_mng_df.marketDataObj.getDataFrame()
Download the following data from Quandal:
CHRIS/CME_SP1
Where CHRIS is the Quandal Data Provider, and the name after the / is the Futures symbol.
The format of the JSON to download from Quandal is:
{ quandal-path :
[ description of data,
[ columns to extract from the returned Quandal DataFrame ],
{ "Quandal_Column" : "Rename Quandal Column" }
]
}
Note:
1. The nested dictionary to rename the original downloaded Quandal data, it is provided since Quandal does
not have standard output column names.
- Futures data from different data sources might have different returned column names for the same
state, that is "Close" vs. "close", etc.
- For sharing the same market data manager object it is recommended to rename the columns to have
standard names, and match the ones provided by Yahoo Finance
2. Yahoo Finance returns standard equity column names
3. All implemented AI and Technical Models used standard column names (same as Yahoo Finance)
Columns: Adj Close, Close, Open, High, Low, Volume
Index: Datetime
4. In order to use the implemented models the Quandal Market Data or any Custom Market Data must have
similar named columns, or the user can implement his own models and use whatever column names he/she
prefers.
5. In this example, the Quandal data source does not provide all standard column names, only open,
which is re-named to 'Adj Close'. Therefore only some of the built models will be available,
as they required High, Low, such as Bollinger Bands
indexQuantCodeMap = {
"CHRIS/CME_S2" : ["SoyBeans", ["Open", "Volume"], { "Open" : "Adj Close" } ],
"CHRIS/ICE_CT1" : ["Cotton", ["Open", "Volume"], { "Open" : "Adj Close" } ],
"CHRIS/CME_SP1" : ["S&P_500", ["Open", "Volume"], { "Open" : "Adj Close" } ] }
dirPathMarketDataQuandl = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/TSMarketDat'
beg_date_for_feature_creation = "2000-10-01"
beg_date_str = "2001-01-01"
end_date_str = "2010-12-31"
q_stock_index_list = list ( indexQuantCodeMap.keys() )
ts_marketdat_mng_quandal = TradeStrategyMarketDataMng(\
marketDataProviderName = "quandl",
supportedTradeAssets = q_stock_index_list,
stock_price_col_name = "Adj Close",
fileNameMarketData = "Quandl_Test_1_PorfolioRegisterItem_Portf_J",
dirPathMarketData = dirPathMarketDataQuandl,
beg_date_str = beg_date_for_feature_creation,
end_date_str = end_date_str,
interval = "1d" ,
updateDataPassEndDate = False,
forceCreateMarketData = False,
indexQuantCodeMap = indexQuantCodeMap # required for quandl market data provider, defined above
)
ts_marketdat_mng_quandal.marketDataObj.getDataFrame()
For AI models, shared pipeline
ts_model_pipline_mng_yf = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng_yf,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_YF",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
ts_model_pipline_mng_df = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng_df,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_DF",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
Note:
- The rnn pipeline builder is built explicitily and the columns to use in the AI RNN based models defined.
- The Quandal data source in the example only has data for the 'Adj Close', 'Volume', the default is
to assume all standard columns for equities returned by Yahoo Finance
ai_rnn_pipeline_builder = WaveNet_RNN_AI_PipelineBuilder(\
stock_index_col = 'Index',
pipeline_cols = ['Adj Close', 'Volume' ],
y_feature_col = 'Adj Close' )
ts_model_pipline_mng_quandal = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng_quandal,
modelPipelineBuilder = ai_rnn_pipeline_builder,
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_Quandal",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
taRegisterItems = [ ]
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 20, 'win_slow' : 100 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_3",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 10, 'win_slow' : 50 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #4
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_4",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_5",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 75 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #6
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_6",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 7, 'win_slow' : 35 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 2 } },
},
} )
# TS #7
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_7",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 7, 'win_slow' : 35 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : True },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #9
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #10
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_macd",
'actions_conf' : { "entry_exit" :
{ 'model' : "macd",
'type' : "ta",
'obj_args' : { 'window_slow' : 26, 'window_fast' : 12, 'window_sign' : 9,
'enter_long_macd_diff' : 0.5 , 'exi_long_macd_diff' : 0.5,
'enter_short_macd_diff' : -0.5, 'exi_short_macd_diff' : -0.5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #11
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_rsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "rsi",
'type' : "ta",
'obj_args' : { 'rsi_enter_long' : 30.0, 'rsi_exit_long' : 30.0,
'rsi_enter_short' : 70.0, 'rsi_exit_short' : 70.0 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng_yf,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_yf )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng_yf,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_yf )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng_yf,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in q_stock_index_list :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng_quandal,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_quandal )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng_quandal,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_quandal )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng_quandal,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in [ '^DJI', '^IXIC' ] :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng_df,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_df )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng_df,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_df )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng_df,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager, Quandal, YahooFinance, DF",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio Inv. Manager, Quandal, YahooFinance, DF",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
session.add( portf_registered_obj )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
session.expunge_all()
del portf_registered_obj
for tsObj in session.query(PortfolioRegisterItem):
print(tsObj.name)
if tsObj.name == "Portfolio Inv. Manager, Quandal, YahooFinance, DF" :
break
portf_registered_obj = tsObj
print(portf_registered_obj.id, portf_registered_obj.name)
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2000-10-05",
strBegDate = "2001-01-01",
strEndDate = "2006-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2006-07-01",
strBegDate = "2007-01-01",
strEndDate = "2008-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2008-07-01",
strBegDate = "2009-01-01",
strEndDate = "2009-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecution( \
name = "Backtest for Portfolio J",
timePeriod = testing_period,
fitDatasets = fit_datasets,
trade_asset_registered_item = portf_registered_obj )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("CHRIS")', engine='python')
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
A type of backtests that are designed to simulate fitting AI Models to different date ranges or to shift the date ranges for a backtest through time.
- Description :
- Re-fit the AI models that a given trade strategy depends on during a backtest
- Scenario:
- Given the following initial setting for a backtest:
AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-06-30
Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2008-12-31
- Re-fit the models after the backtest has executed for a period (n days, n weeks, n months), by
either shifting the Training and Validation period or extending, that is anchoring the Training
Period start date
Example 1
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-06-30
1. Training Period:
1. Begin Date = 2001-04-01
2. End Date = 2006-03-31
2. Validation Period:
1. Begin Date = 2006-04-01
2. End Date = 2007-09-30
Example 2
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-06-30
1. Training Period:
1. Begin Date = 2001-01-01
2. End Date = 2006-03-31
2. Validation Period:
1. Begin Date = 2006-04-01
2. End Date = 2007-09-30
...
Description:
Backtest:
1. Type: WFA Backtest
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2000-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-12-31
4. Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2008-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "1980-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_From_1980",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
MarketData.getDateRangesForStocks( ts_marketdat_mng.marketDataObj.getDataFrame() )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_3",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_5",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = None )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = None )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Multiple-LSTM TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with Multiple-LSTM and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "1999-07-01",
strBegDate = "2000-01-01",
strEndDate = "2005-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2008-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecutionWFA( \
name = "WFA Backtest for subset of Portfolio D - non-anchored training",
test_period = testing_period,
fit_datasets_start = fit_datasets,
trade_asset_registered_item = portf_registered_obj,
wfa_slide_win_size = 4,
wfa_slide_win_period = WFASlideWindowPeriod.Months,
anchor_fit_dataset = False )
tsExecuteEngine.evaluate()
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']]
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
tsExecuteEngine.getSummaryMetrics().T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Description:
Backtest:
1. Type: WFA Backtest
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2000-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-12-31
4. Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2010-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "1980-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_From_1980",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_model_pipline_mng_g = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_G",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #10
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #16
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_kama",
'actions_conf' : { "entry_exit" :
{ 'model' : "kama",
'type' : "ta",
'obj_args' : { 'window_fast' : 10, 'pow1_fast' : 2, 'pow2_fast' : 30,
'window_slow' : 10, 'pow1_slow' : 5, 'pow2_slow' : 30 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #17
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_tsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "tsi",
'type' : "ta",
'obj_args' : { 'window_slow' : 25, 'window_fast' : 5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_g )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_g )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - TechAnal & AI Trading Models TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with TechAnal & AI Trading Models, and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "1999-07-01",
strBegDate = "2000-01-01",
strEndDate = "2005-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2010-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecutionWFA( \
name = "WFA Backtest for subset of Portfolio G - anchored training-set",
test_period = testing_period,
fit_datasets_start = fit_datasets,
trade_asset_registered_item = portf_registered_obj,
wfa_slide_win_size = 6,
wfa_slide_win_period = WFASlideWindowPeriod.Months,
anchor_fit_dataset = True )
tsExecuteEngine.evaluate()
DOW Jones
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("DJI")', engine='python')
Nasdaq
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("IXIC")', engine='python')
S&P 500
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("GSPC")', engine='python')
Dow Jones
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^DJI" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^DJI" ').T.head(60)
Nasdaq
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^IXIC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^IXIC" ').T.head(60)
S&P 500
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^GSPC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^GSPC" ').T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Description:
Backtest:
1. Type: WFA Backtest
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2003-01-01
2. End Date = 2008-12-31
2. Validation Period:
1. Begin Date = 2009-01-01
2. End Date = 2010-12-31
4. Test Period:
1. Begin Date = 2011-01-01
2. End Date = 2012-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "1980-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_From_1980",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_model_pipline_mng_g = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_G",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #10
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #16
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_kama",
'actions_conf' : { "entry_exit" :
{ 'model' : "kama",
'type' : "ta",
'obj_args' : { 'window_fast' : 10, 'pow1_fast' : 2, 'pow2_fast' : 30,
'window_slow' : 10, 'pow1_slow' : 5, 'pow2_slow' : 30 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #17
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_tsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "tsi",
'type' : "ta",
'obj_args' : { 'window_slow' : 25, 'window_fast' : 5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_g )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_g )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - TechAnal & AI Trading Models TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with TechAnal & AI Trading Models, and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2002-07-01",
strBegDate = "2003-01-01",
strEndDate = "2008-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2008-07-01",
strBegDate = "2009-01-01",
strEndDate = "2010-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2010-07-01",
strBegDate = "2011-01-01",
strEndDate = "2012-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecutionWFA( \
name = "WFA Backtest-3 for subset of Portfolio G - anchored training-set",
test_period = testing_period,
fit_datasets_start = fit_datasets,
trade_asset_registered_item = portf_registered_obj,
wfa_slide_win_size = 6,
wfa_slide_win_period = WFASlideWindowPeriod.Months,
anchor_fit_dataset = False )
tsExecuteEngine.evaluate()
DOW Jones
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("DJI")', engine='python')
Nasdaq
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("IXIC")', engine='python')
S&P 500
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("GSPC")', engine='python')
Dow Jones
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^DJI" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^DJI" ').T.head(60)
Nasdaq
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^IXIC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^IXIC" ').T.head(60)
S&P 500
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^GSPC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^GSPC" ').T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
- Description :
- Run a set of backtests by shifting the Training, Validation and Test Periods through time, or
increasing the duration of the Training and Validation Periods through time and shifting only
the Test Period
- Purpose of these set of backtest is to analysis the out-of-sample performance for trade strategies
and/or portfolios through time
- The sliding-window of the training, validation, and test periods are configurable.
- This type of backtest is combined with the regular WFA backtest, which allows to re-fit
models during each sliding window shift per the configuration of the regular WFA Backtest.
- This type of backtest can be thought of as double sliding window, one sliding window for shifting or
increasing the size of the training and validation periods, the second one to control how often
to re-fit models during a backtest.
- Due to the potential large number of backtests that can be executed, only the metadata, and summary
statistics and performance data are stored per backtest.
- All backtest statistical and metadata summary information is stored in a dataframe
- The metadata information stored includes actual Fit Data Ranges, Test Period, and re-fit
frequency during a backtest
- Example A:
- Scenario 1:
- Fix Fit and Test Periods.
- Given the following initial setting for a backtest:
Fit Dataset:
train: 1/1/2000 to 12/31/2004
validation: 1/1/2005 to 12/31/2005
Test Period: 1/1/2006 to 12/31/2006
- Run WFA backtests with the datasets above for different re-fit set frequencies, non-anchored:
- 2 weeks
- 1 month
- 3 month
- This example will run the WFA defined part 1, three-times, re-fitting the AI models each time
per the frequency defined.
- Example B:
- Scenario 2:
- WFA Scenario implementation that slides through time the Fit datasets as well as the test period.
- Given the following initial setting for a backtest:
Fit Dataset:
train: 1/1/2000 to 12/31/2004
validation: 1/1/2005 to 12/31/2005
Test Period: 1/1/2006 to 12/31/2006
- Run WFA backtests from the starting datasets above, incrementing sliding window of the datasets
for the Fit and Test periods by one-year, while the end date for the test period is less than
or equal to 12/31/2020
- Re-fit the AI Models in a backtest every 6 months, don't anchor the training set
- Each Iteration is one complete backtest, and in each backtest the AI Models are fitted at
the start and at July 1 of the current backtest year.
Iteration 1:
train: 1/1/2000 to 12/31/2004
validation: 1/1/2005 to 12/31/2005
Test Period: 1/1/2006 to 12/31/2006
Iteration 2:
train: 1/1/2001 to 12/31/2005
validation: 1/1/2006 to 12/31/2006
Test Period: 1/1/2007 to 12/31/2007
..
..
Iteration Last:
train: 1/1/2014 to 12/31/2018
validation: 1/1/2019 to 12/31/2019
Test Period: 1/1/2020 to 12/31/2020
- Example C:
- Scenario 3:
- WFA Scenario implementation that increases through time the date range of the Fit datasets,
but not the test period, as the fit dataset is anchored to its initial position.
- The train set is anchored, and the train and validation set are expanded in every iteration,
pushing the test period dataset forward in time, but with the same length.
- The increased per iteration of the train and validation set are configurable, as well as
the maximum end date for the test period.
- Given the following initial setting for a backtest:
Fit Dataset:
train: 1/1/2000 to 12/31/2004
validation: 1/1/2005 to 12/31/2005
Test Period: 1/1/2006 to 12/31/2006
- Run WFA backtests from the starting datasets above, incrementing sliding window of the datasets
for the Fit and Test periods by one-year, while the end date for the test period is less than
or equal to 12/31/2020
- Re-fit the AI Models in a backtest every 6 months, don't anchor the training set
- This class will iterate through time by creating updated Fit datsets, and Test Periods, with
one year increments overall to the train and validation period. Train dataset increased by 9 months
per iteration, and Validation dataset by 3 months, the test period is just shifted by the one-year.
- that is:
* Iteration 1:
train: 1/1/2000 to 12/31/2004
validation: 1/1/2005 to 12/31/2005
Test Period: 1/1/2006 to 12/31/2006
* Iteration 2:
train: 1/1/2000 to 9/30/2005 (date range increased by nine-months)
validation: 10/1/2005 to 12/31/2006 (date range increaed by three-months and
shifted by nine-months)
Test Period: 1/1/2007 to 12/31/2007 (date range shifted by one-year)
* Iteration 3:
train: 1/1/2000 to 6/30/2006 (date range increased by nine-months)
validation: 7/1/2006 to 12/31/2007 (date range increaed by three-months and
shifted by nine-months)
Test Period: 1/1/2008 to 12/31/2008 (date range shifted by one-year)
..
..
* Iteration Last:
train: 1/1/2000 to xxx
validation: yyy to 12/31/2019
Test Period: 1/1/2020 to 12/31/2020
Description:
Backtest:
1. Type: Slide Window WFA Backtest
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2000-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-12-31
3. Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2008-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "1980-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_From_1980",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
MarketData.getDateRangesForStocks( ts_marketdat_mng.marketDataObj.getDataFrame() )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_3",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_5",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = None )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = None )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Multiple-LSTM TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with Multiple-LSTM and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "1999-07-01",
strBegDate = "2000-01-01",
strEndDate = "2005-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2008-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecutionSlideWindowWFA( \
name = "SW WFA Backtest for subset of Portfolio D - non-anchored training",
trade_asset_registered_item = portf_registered_obj,
wfa_slide_win_size_list = [ 3, 6] , # list of int, associated with wfa_slide_win_period_list
wfa_slide_win_period_list = [ WFASlideWindowPeriod.Months, WFASlideWindowPeriod.Months ],
wfa_sw_scenario_obj = WFASlideWinScenarioFixDatasets(fit_dataset = fit_datasets,
test_period = testing_period ),
anchor_fit_dataset = False )
tsExecuteEngine.evaluate()
tsExecuteEngine.getSummaryMetrics().info()
tsExecuteEngine.getSummaryMetrics()\
[ [ 'wfa_sw_iter_no', 'wfa_slide_win_size', 'wfa_slide_win_period', 'tr_period_len_days', 'val_period_len_days',
'test_period_len_days', 'ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset',
'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Description:
Backtest:
1. Type: Slide Window WFA Backtest
2. Start AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2000-01-01
2. End Date = 2005-12-31
2. Validation Period:
1. Begin Date = 2006-01-01
2. End Date = 2007-12-31
3. Final AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2004-01-01
2. End Date = 2009-12-31
2. Validation Period:
1. Begin Date = 2010-01-01
2. End Date = 2011-12-31
4. Start Test Period:
1. Begin Date = 2008-01-01
2. End Date = 2008-12-31
5. Final Test Period:
1. Begin Date = 2012-01-01
2. End Date = 2012-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "1980-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_From_1980",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_3",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
# TS #5
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm_5",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
},
} )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = None )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = None )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - Multiple-LSTM TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with Multiple-LSTM and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "1999-07-01",
strBegDate = "2000-01-01",
strEndDate = "2005-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2008-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecutionSlideWindowWFA( \
name = "SW WFA Backtest-2 for subset of Portfolio D - non-anchored training",
trade_asset_registered_item = portf_registered_obj,
wfa_slide_win_size_list = [ 6 ] , # list of int, associated with wfa_slide_win_period_list
wfa_slide_win_period_list = [ WFASlideWindowPeriod.Months ],
wfa_sw_scenario_obj = WFASlideWinScenarioSlidingDatasets(\
fit_dataset = fit_datasets,
test_period = testing_period,
slide_win_size = 1,
slide_win_period = WFASlideWindowPeriod.Years,
end_date_last_test_period = datetime(year=2012, month=12, day=31) ),
anchor_fit_dataset = False )
tsExecuteEngine.evaluate()
tsExecuteEngine.getSummaryMetrics()\
[ [ 'wfa_sw_iter_no', 'wfa_slide_win_size', 'wfa_slide_win_period', 'tr_period_len_days', 'val_period_len_days',
'test_period_len_days', 'ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset',
'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ]
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
Description:
Backtest:
Type: Slide Window WFA Backtest
Iteration 1:
train: 1/1/2000 to 12/31/2005
validation: 1/1/2006 to 12/31/2007
Test Period: 1/1/2008 to 12/31/2008
Iteration 2:
train: 1/1/2000 to 9/30/2006 (date range increased by nine-months)
validation: 10/1/2006 to 12/31/2008 (date range increased by three-months and shifted
by nine-months)
Test Period: 1/1/2009 to 12/31/2009 (date range shifted by one-year)
Iteration 3:
train: 1/1/2000 to 6/30/2007 (date range increased by nine-months)
validation: 7/1/2007 to 12/31/2009 (date range increaed by three-months and shifted
by nine-months)
Test Period: 1/1/2010 to 12/31/2010 (date range shifted by one-year)
...
...
Iteration Last:
train: 1/1/2000 to xxx
validation: yyy to 12/31/2011
Test Period: 1/1/2012 to 12/31/2012
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "1980-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_From_1980",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_model_pipline_mng_g = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_G",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #10
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #16
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_kama",
'actions_conf' : { "entry_exit" :
{ 'model' : "kama",
'type' : "ta",
'obj_args' : { 'window_fast' : 10, 'pow1_fast' : 2, 'pow2_fast' : 30,
'window_slow' : 10, 'pow1_slow' : 5, 'pow2_slow' : 30 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #17
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_tsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "tsi",
'type' : "ta",
'obj_args' : { 'window_slow' : 25, 'window_fast' : 5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_g )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_g )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - TechAnal & AI Trading Models TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with TechAnal & AI Trading Models, and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "1999-07-01",
strBegDate = "2000-01-01",
strEndDate = "2005-12-31" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2005-07-01",
strBegDate = "2006-01-01",
strEndDate = "2007-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2007-07-01",
strBegDate = "2008-01-01",
strEndDate = "2008-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecutionSlideWindowWFA( \
name = "SW WFA Backtest-3 for subset of Portfolio G - anchored training",
trade_asset_registered_item = portf_registered_obj,
wfa_slide_win_size_list = [ 6 ] , # list of int, associated with wfa_slide_win_period_list
wfa_slide_win_period_list = [ WFASlideWindowPeriod.Months ],
wfa_sw_scenario_obj = WFASlideWinScenarioAnchoredSlidingDatasets(\
fit_dataset = fit_datasets,
test_period = testing_period,
train_slide_win_size = 9, # increase train set 9 months every sw incr
val_slide_win_size = 3, # increase val set 3 months every sw incr
# and shift by 9 months
slide_win_period = WFASlideWindowPeriod.Months,
end_date_last_test_period = datetime(year=2012, month=12, day=31) ),
anchor_fit_dataset = True )
tsExecuteEngine.evaluate()
tsExecuteEngine.getSummaryMetrics()\
[ [ 'wfa_sw_iter_no', 'wfa_slide_win_size', 'wfa_slide_win_period', 'tr_period_len_days', 'val_period_len_days',
'test_period_len_days', 'ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset',
'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^DJI" ')
tsExecuteEngine.getSummaryMetrics()\
[ [ 'wfa_sw_iter_no', 'wfa_slide_win_size', 'wfa_slide_win_period', 'tr_period_len_days', 'val_period_len_days',
'test_period_len_days', 'ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset',
'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^IXIC" ')
tsExecuteEngine.getSummaryMetrics()\
[ [ 'wfa_sw_iter_no', 'wfa_slide_win_size', 'wfa_slide_win_period', 'tr_period_len_days', 'val_period_len_days',
'test_period_len_days', 'ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset',
'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^GSPC" ')
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
The reports currently supported are provided by quantstats, https://github.com/ranaroussi/quantstats.
from trade_engine.reports.BacktestReports import BacktestReports
Description:
Backtest:
1. Type: WFA Backtest
2. AI Models Fit Date Ranges
1. Training Period:
1. Begin Date = 2008-05-01
2. End Date = 2010-04-30
2. Validation Period:
1. Begin Date = 2010-05-01
2. End Date = 2010-12-31
4. Test Period:
1. Begin Date = 2011-01-01
2. End Date = 2012-12-31
portfolio_trade_assets = [ "^DJI", "^IXIC", "^GSPC" ]
dirPathMarketData = '/Users/gustavozambrana/Google Drive/Python/Projects/Trade_Engine/Data/US_Market_Indixes'
supportedTradeAssets = [ "^DJI", "^IXIC", "^GSPC", "UDOW", "SDOW", "SPXL", "SPXS", "TQQQ", "SQQQ" ]
beg_date_str = "1980-01-01"
end_date_str = None
ts_marketdat_mng = TradeStrategyMarketDataMng(\
marketDataProviderName = "yahoofinance",
supportedTradeAssets = supportedTradeAssets,
stock_price_col_name = "Adj Close",
fileNameMarketData = "US_Indixes_And_ETFs_MrkData_From_1980",
dirPathMarketData = dirPathMarketData,
beg_date_str = beg_date_str,
end_date_str = end_date_str,
interval = "1d" )
ts_model_pipline_mng_g = TradeStrategyModelPipelineDataMng(\
tsMarketDataMng = ts_marketdat_mng,
package = "trade_engine.model.ai.WaveNet_RNN_AI_PipelineBuilder",
model_class_name = "WaveNet_RNN_AI_PipelineBuilder",
fileNameMarketData = "WaveNet_RNN_AI_Pipeline_Data_Portf_G",
forceCreatePipelineData = True,
dirPathMarketData = dirPathMarketData )
brokerTradingObj = SimBrokerTrading()
tsConfDat = TradeStrategyConfigData()
# Set Config
tsConfDat.tsSetConfig(\
ts_set_conf_dict =
{ 'set_actions_conf' :
{ "entry" : { 'votingPolicy' : 1, 'incNoInvestInVotingDec' : False,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1, 'cross_over_signal_for_trading' : False },
"exit" : { 'votingPolicy' : 1,
'minPerVotingLong' : 0.0, 'minPerVotingShort' : 0.0,
'lastn_match' : 1 }
}
} )
# TS #1
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_1",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 } ,
'entry_policy_args' : { 'lastn_match' : 1, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 } },
"stop_lost" :
{ 'model' : "fix_perc",
'type' : "ta",
'obj_args' : { 'long_stop_los_per' : 5, 'short_stop_los_per' : 5 } }
},
} )
# TS #2
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_lstm",
'actions_conf' : { "entry_exit" :
{ 'model' : "lstm",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #3
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_conv1d_gru",
'actions_conf' : { "entry_exit" :
{ 'model' : "conv1d_gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 50, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 5,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #8
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_gru_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "gru",
'type' : "ai",
'obj_args' : { 'n_steps' : 40, 'no_neurons_lay1' : 20,
'no_neurons_lay2' : 20, 'pred_ahead_periods' : 3,
'ext_trade_dyn' : True, 're_eval_exit_dyn' : True,
'epochs' : 20 }
}
} } )
# TS #10
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_sma_2",
'actions_conf' : { "entry_exit" :
{ 'model' : "sma",
'type' : "ta",
'obj_args' : { 'win_fast' : 15, 'win_slow' : 60 } ,
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #16
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_kama",
'actions_conf' : { "entry_exit" :
{ 'model' : "kama",
'type' : "ta",
'obj_args' : { 'window_fast' : 10, 'pow1_fast' : 2, 'pow2_fast' : 30,
'window_slow' : 10, 'pow1_slow' : 5, 'pow2_slow' : 30 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
# TS #17
tsConfDat.addTSConf(\
ts_conf_dict = {
'ts_unique_tag' : "ts_tsi",
'actions_conf' : { "entry_exit" :
{ 'model' : "tsi",
'type' : "ta",
'obj_args' : { 'window_slow' : 25, 'window_fast' : 5 },
'entry_policy_args' : { 'lastn_match' : 2, 'cross_enter_signal' : False },
'exit_policy_args' : { 'lastn_match' : 1 },
},
} } )
taRegisterItems = [ ]
tsActionGen = TradeStrategyActionGenerator()
for trade_asset in portfolio_trade_assets :
num_ts = len(tsConfDat.name_list) + 1 # adding the composite trade strategy
for i in range( num_ts ) :
if i <= num_ts - 2:
# *** Trade Strategy
actionDatItem = tsConfDat.getTSConfigDataForItem(item_num = i )
ts_unique_name = actionDatItem.name_list[0]
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = actionDatItem,
pipelineMng = ts_model_pipline_mng_g )
else :
# Composite Trade Strategy
ts_unique_name = "all"
trade_enter_set, trade_exit_set = tsActionGen.createEnterAndExitActionObjs ( \
ts_marketdat_mng = ts_marketdat_mng,
test_trade_asset = trade_asset,
dirPathMarketData = dirPathMarketData,
actionData = tsConfDat,
pipelineMng = ts_model_pipline_mng_g )
ts_model = TradeStrategyBaseModelPersist(\
trade_asset = trade_asset,
ts_marketdat_mng = ts_marketdat_mng,
ts_enter = trade_enter_set,
ts_exit = trade_exit_set,
model_type = ts_unique_name,
model_version = "1.0",
market_data_period = MarketDataPeriod.Day,
)
# *** Trade Strategy Investment Manager
taInvMng = TradingAssetInvMngBase(\
trade_asset = trade_asset,
minPeriodsBetweenTrades = 1,
initInvestment = 1000000.0,
capPercentToInvestPerTrade = 100.0,
minInvestPercent = 5.0,
min_account_balance = 1000.0 )
# *** Trade Register Item
# *** This object has all configuration defined to execute a given trade strategy,
# *** It can be executed individually in backtest or as part of a portfolio
ta_registered_item = TradingAssetRegisterItem(\
trade_name = "{0}_{1}".format(ts_unique_name, trade_asset),
ts_model = ts_model,
tradingExecutionType = TradingAssetExecutionType.Simulation,
tradingBrokerObj = brokerTradingObj,
tradingAssetInvMngObj = taInvMng )
taRegisterItems.append(ta_registered_item)
print("Number of configured Trade Asset Register Items: ", len(taRegisterItems) )
# *** Porttfolio Investment Manager across all individual trade strategies investment managers
# *** Role:
# *** 1) Managed investment funds across n number of trade strategies
# *** 2) Rebalance portfolio periodically or never
# *** 3) Portfolio Policy Weights
# *** equal weights (default)
# *** MinimumVariance
# *** User defined
# *** 4) Track Portfolio level returns and metrics
# *** Individual Trade Strategies return and metrics tracked through their
# *** investment managers
# *** Portfolio funds are split across each trade strategy investment manager based on the
# *** portfolio weights
portfInvMng = PortfolioInvMng(\
name = "Portfolio Inv. Manager for US-Indexes - TechAnal & AI Trading Models TS",
initInvestment = 2000000.0 * ( len(taRegisterItems) + 3), # 200K/trade-strag
portfRebalanceFreq = PortfolioRebalanceFrequency.Never,
portfRebalancePeriods = 1,
minInvestPercent = 5.0,
min_account_balance = 1000.0,
minPeriodsBetweenTrades = 1,
capPercentToInvestPerTrade = 100.0 )
# *** Set of Trade Register Items
#
# Configuration Summary:
# - Each set has 1 to n trade register items as previously described
# - Optionally add long only trade strategies for all unique trade assets in the
# portfolio
# - (Dow, Nasdaq, and S&P 500 for this portfolio) in this example
# - Might be useful for bench marking trade strategies
# - Each set has one Investment Portfolio Manager
# - portfolio funds allocated per portfolio policy rules and re-balanced (if defined),
# across all trade strategies
portf_registered_obj = PortfolioRegisterItem(\
name = "Portfolio of US-Indexes with TechAnal & AI Trading Models, and Long strategies",
taRegisterItems = taRegisterItems,
portfolioInvMngObj = portfInvMng,
tradingExecutionType = TradingAssetExecutionType.Simulation,
add_long_only_ts = True )
In Order to backtest a trade strategy or portfolio that has AI models, the following data is required:
Training Date Range
Validation Date Range
Test Period Date Range
Date Ranges are defined with a DatasetDef object, and have the following parameters:
Note:
training_period = DatasetDef(datasetType = DatasetType.Training,
strBegDateAIModels = "2007-11-01",
strBegDate = "2008-05-01",
strEndDate = "2010-04-30" )
validation_period = DatasetDef(datasetType = DatasetType.Validation,
strBegDateAIModels = "2009-11-01",
strBegDate = "2010-05-01",
strEndDate = "2010-12-31" )
testing_period = DatasetDef(datasetType = DatasetType.Testing,
strBegDateAIModels = "2010-07-01",
strBegDate = "2011-01-01",
strEndDate = "2012-12-31" )
fit_datasets = FitDatasetsDef( trainingDataset = training_period,
validationDataset = validation_period )
tsExecuteEngine = BacktestExecutionWFA( \
name = "WFA Backtest-4 for subset of Portfolio G - anchored training-set",
test_period = testing_period,
fit_datasets_start = fit_datasets,
trade_asset_registered_item = portf_registered_obj,
wfa_slide_win_size = 3,
wfa_slide_win_period = WFASlideWindowPeriod.Months,
anchor_fit_dataset = False )
tsExecuteEngine.evaluate()
DOW Jones
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("DJI")', engine='python')
Nasdaq
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("IXIC")', engine='python')
S&P 500
tsExecuteEngine.getExecutionMetrics()[ ['TA_Reg_Name', 'Is_Portfolio', 'Num_Trades_Completed',
'Init_Capital', 'Final_Capital', 'Cum_Return', 'Backtest_Status']].\
query('TA_Reg_Name.str.contains("GSPC")', engine='python')
Dow Jones
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^DJI" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^DJI" ').T.head(60)
Nasdaq
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^IXIC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^IXIC" ').T.head(60)
S&P 500
tsExecuteEngine.getSummaryMetrics()\
[ ['ta_reg_item_name', 'is_portfolio', 'model_type', 'trade_asset', 'Start Period', 'End Period', 'Time in Market ',
'Cumulative Return ', 'CAGR%', 'Sharpe', 'Max Drawdown ', 'Best Day ', 'Worst Day '] ].\
query('trade_asset == "^GSPC" ')
tsExecuteEngine.getSummaryMetrics().query('trade_asset == "^GSPC" ').T.head(60)
session.add( tsExecuteEngine )
try :
session.commit()
except Exception as inst:
print(type(inst)) # the exception instance
print(inst)
print(inst.args) # arguments stored in .args
session.rollback() # roll back changes
finally:
print(" done")
BacktestReports.createTearsheetReport(\
backtest = tsExecuteEngine,
ta_reg_item_name = "all_^IXIC",
ta_reg_item_benchmark_name = "Long-Only Trade Strategy for ^IXIC ", # optional
report_type = "full" )
BacktestReports.createTearsheetReport(\
backtest = tsExecuteEngine,
ta_reg_item_name = "all_^DJI",
ta_reg_item_benchmark_name = "Long-Only Trade Strategy for ^DJI ", # optional
report_type = "full" )
BacktestReports.createTearsheetReport(\
backtest = tsExecuteEngine,
ta_reg_item_name = "Portfolio of US-Indexes with TechAnal & AI Trading Models, and Long strategies",
ta_reg_item_benchmark_name = None, # optional
report_type = "full" )
BacktestReports.createHTMLTearsheetReport(\
backtest = tsExecuteEngine,
ta_reg_item_name = "all_^IXIC",
ta_reg_item_benchmark_name = "Long-Only Trade Strategy for ^IXIC ", # optional
title = 'Strategy Tearsheet',
download_filename = 'ts-tearsheet.html' )